Monorepo structure #2300

gchoqueux · 2024-03-21T14:50:56Z

Description

Refactoring iTowns to monorepo structure.

The goal is to split itowns down into feature packages.
As an example, I've started with the geographic functionalities. (@itowns/geographic)

For the moment Debug and Widget modules are private.

Motivation and Context

Split code in packages for clearly structured code.
Simplifies development and facilitates contributions.
Increases the scope of users who only want to use a few functions.
This structure makes it necessary to make functions independent

Code movement

The classes Coordinates, Crs, Ellipsoid, Extent, OrientationUtils files are moved to ./packages/geographic/src.
The proj4 dependency moves to geographic package.

The unit tests moves to packages/xxx/test.
the code utils/debug moves packages/Debug
the code src/Utils/Gui moves packages/Widget

dependencies

This proposal from monorepo doesn't use monorepo package (by example Lerna).
New dev dependency is Concurrently, it's used to watch script in parallel
Itowns dependencies (three, proj4...) are moved to the packages that use them

node scripts

The scripts uses the options --workspaces or -ws , to call the corresponding script in each sub packages.

gchoqueux · 2024-03-27T17:54:40Z

@Desplandis I resolve some issues

resolve module with exportsFields option
remove lerna with --workspace cli option

Desplandis · 2024-03-27T20:00:17Z

@gchoqueux Nice! I will take a quick look of your changes and write a todo list of things that should absolutely be tested before reviewing more thoroughly this PR. Expect it for tomorrow or next tuesday!

@jailln @mgermerie @ftoromanoff @AnthonyGlt I think we'll need your inputs on this change! ;)

This should be a good start to fix #2197, #1930, #2201 (list not exhaustive, feel free to complete). Shall we create a meta-issue to aggregate all those issues?

Desplandis

Hi! This is first batch of my review of this PR. Unfortunately I'm kinda blocked on testing the distribution since the compiled code for the itowns package is incorrect (especially with imports to relative modules).

Moreover, there is a lot of points that I would like you to address. I'll resume the review once those points have been resolved.

Since this PR is massive, @jailln there may be some things that eluded me, do not hesitate to complement this review. =)

Monorepo_Roadmap.md

.gitignore

Gruntfile.cjs

package-lock.json

packages/Main/.eslintignore

Desplandis · 2024-11-07T12:32:41Z

packages/Main/package.json

+    "test-with-coverage_lcov": "c8 -n src --reporter=lcov cross-env npm run test-unit",
+    "watch": "cross-env BABEL_DISABLE_CACHE=1 babel --watch src --out-dir lib",
+    "postinstall": "cross-env NO_UPDATE_NOTIFIER=true node ./scripts/prepare.mjs && node ./scripts/replace.config.mjs",
+    "prepublishOnly": "npx copyfiles -u 1 \"../../examples/**/*\" ./examples/ && npx copyfiles -u 1 \"../../docs/**/*\" ./docs/ && npx copyfiles -u 1 \"../../dist/**/*\" ./dist/ ",


Or just not distribute examples and docs like most other packages.

to be decided

packages/Widget/.babelrc

gchoqueux · 2024-11-13T17:30:49Z

Hi! This is first batch of my review of this PR. Unfortunately I'm kinda blocked on testing the distribution since the compiled code for the itowns package is incorrect (especially with imports to relative modules).

Moreover, there is a lot of points that I would like you to address. I'll resume the review once those points have been resolved.

Since this PR is massive, @jailln there may be some things that eluded me, do not hesitate to complement this review. =)

@Desplandis

I've finished answering/fixing all your comments.
I've also corrected the transpiling for the tests.

jailln

Not much to add to @Desplandis review.

I just think we should explain this new structure in the README and add sub package installation possibilities.

Also, how does it work with releasing and versioning? does subpackage all have the same version or can we (should we?) release them independently ?

packages/Geographic/README.md

package.json

gchoqueux · 2024-11-28T12:38:40Z

I just think we should explain this new structure in the README and add sub package installation possibilities.

Yes, I'll document

Also, how does it work with releasing and versioning? does subpackage all have the same version or can we (should we?) release them independently ?

In a monorepo, it's more complicated to manage a different version for each module, but if we can find a simple and robust solution we can implement it later.

Desplandis · 2024-12-02T15:11:41Z

@gchoqueux Did you try to publish it in your own npm packages? I tried to do it with a custom alias, but didn't manage to make it working...

This is the a log of my last attempt:
https://github.com/Desplandis/itowns/actions/runs/12121853265/job/33793853058

The CI works correctly but I would like to ensure that both npm and website publishing are working.

jailln

Last batch of comments for me, only documentation remarks, I let you see with @Desplandis for package publishing. Thanks for the changes you already made

jailln · 2024-12-03T08:18:24Z

packages/Geographic/README.md

+The geographic package provides utilities for handling coordinates, ellipsoids, extents and rotations across different coordinate systems.
+
+* [Coordinates](http://www.itowns-project.org/itowns/docs/#api/Geographic/Coordinates) : A Coordinates object (geodetic datum), defined by a [crs] and three values.
+* [Extent](http://www.itowns-project.org/itowns/docs/#api/Geographic/Extent) : Extent is geographical bounding rectangle defined by 4 limits: west, east, south and north.


can you add: and a crs ?

jailln · 2024-12-03T08:19:03Z

packages/Geographic/README.md

+
+The geographic package provides utilities for handling coordinates, ellipsoids, extents and rotations across different coordinate systems.
+
+* [Coordinates](http://www.itowns-project.org/itowns/docs/#api/Geographic/Coordinates) : A Coordinates object (geodetic datum), defined by a [crs] and three values.


can you replace [crs] by Coordinate Reference System (crs)

jailln · 2024-12-03T08:19:25Z

packages/Geographic/README.md

+
+The geographic package provides utilities for handling coordinates, ellipsoids, extents and rotations across different coordinate systems.
+
+* [Coordinates](http://www.itowns-project.org/itowns/docs/#api/Geographic/Coordinates) : A Coordinates object (geodetic datum), defined by a [crs] and three values.


Can you organize this list alphabetically please? and add missing classes and variables (CoordStars and ellipsoidSizes). BTW I would for renaming ellipsoidSizes to WGS84EllipsoidSize while we're at it.

jailln · 2024-12-03T08:24:10Z

packages/Geographic/README.md

+
+* [Coordinates](http://www.itowns-project.org/itowns/docs/#api/Geographic/Coordinates) : A Coordinates object (geodetic datum), defined by a [crs] and three values.
+* [Extent](http://www.itowns-project.org/itowns/docs/#api/Geographic/Extent) : Extent is geographical bounding rectangle defined by 4 limits: west, east, south and north.
+* [Crs](http://www.itowns-project.org/itowns/docs/#api/Geographic/CRS) : This module provides basic methods to manipulate a CRS (as a string). Visiting [epsg.io](https://epsg.io/) to more coordinate system worldwide. Use **Proj4js** definition in epsg.io.


Visiting [epsg.io](https://epsg.io/) to more coordinate system worldwide. Use **Proj4js** definition in epsg.io. -> Visit epsg.io to explore available coordinates systems and use their proj4js definition to define them in itowns with proj4.defs(crs, proj4def)

jailln · 2024-12-03T08:31:01Z

packages/Geographic/README.md

+* [Coordinates](http://www.itowns-project.org/itowns/docs/#api/Geographic/Coordinates) : A Coordinates object (geodetic datum), defined by a [crs] and three values.
+* [Extent](http://www.itowns-project.org/itowns/docs/#api/Geographic/Extent) : Extent is geographical bounding rectangle defined by 4 limits: west, east, south and north.
+* [Crs](http://www.itowns-project.org/itowns/docs/#api/Geographic/CRS) : This module provides basic methods to manipulate a CRS (as a string). Visiting [epsg.io](https://epsg.io/) to more coordinate system worldwide. Use **Proj4js** definition in epsg.io.
+* [OrientationUtils](http://www.itowns-project.org/itowns/docs/#api/Geographic/OrientationUtils) : it provides methods to compute the quaternion that models a rotation defined with various conventions, including between different CRS.


utilities to compute a rotation quaternion from various rotation conventions, including between different crs.

jailln · 2024-12-03T08:35:37Z

packages/Geographic/README.md

+* [Extent](http://www.itowns-project.org/itowns/docs/#api/Geographic/Extent) : Extent is geographical bounding rectangle defined by 4 limits: west, east, south and north.
+* [Crs](http://www.itowns-project.org/itowns/docs/#api/Geographic/CRS) : This module provides basic methods to manipulate a CRS (as a string). Visiting [epsg.io](https://epsg.io/) to more coordinate system worldwide. Use **Proj4js** definition in epsg.io.
+* [OrientationUtils](http://www.itowns-project.org/itowns/docs/#api/Geographic/OrientationUtils) : it provides methods to compute the quaternion that models a rotation defined with various conventions, including between different CRS.
+* Ellipsoid : In geodesy, a reference ellipsoid is a mathematically defined surface that approximates the geoid.


proposition: A representation of an [ellipsoid](https://en.wikipedia.org/wiki/Ellipsoid) and useful computation methods (geodetic normal, etc.)

jailln · 2024-12-03T08:37:55Z

packages/Geographic/README.md

+
+// change projection system to EPSG:2154
+
+// defs EPSG:2154 crs (visiting epsg.io to get new definition and export to Proj4js)


visiting -> visit

to get new definition and export to Proj4js : to get the proj4js definition of a crs

jailln · 2024-12-03T08:41:13Z

packages/Geographic/README.md

+// to the geocentric frame (epsg:4978)
+quat_crs2crs = OrientationUtils.quaternionFromCRSToCRS("EPSG:2154", "EPSG:4978")(origin);
+// Compute the rotation of a sensor platform defined by its attitude
+quat_attitude = OrientationUtils.quaternionFromAttitude(attitude);


thanks for these examples :) can you define a mock attitude above to ease understanding for newbies please?

jailln · 2024-12-03T08:58:42Z

README.md

 ```bash
 npm install --save itowns
 ```

+```js


I propose to replace by something more generic and that will be easier to update when new packages are added and to remove examples since we already have examples in each sub-package documentation:

iTowns is currently moving to a monorepo organization and to a segmentation in sub-modules, allowing to import only some of itowns functionalities. Current itowns sub-modules are: - [@itowns/geographic](packages/Geographic/README.md): `npm install --save @itowns/geographic` - [@itowns/widgets](packages/Widgets/README.md): `npm install --save @itowns/widgets` - [@itowns/core](packages/Main/README.md); `npm install --save @itowns/core`

Note the s at the end of @itowns/widgets since there is more: can you update this package name and also the folder name: packages/Widget -> packages/Widgets

jailln · 2024-12-03T09:01:31Z

packages/Main/README.md

@@ -0,0 +1,131 @@
+![iTowns](https://raw.githubusercontent.com/iTowns/itowns.github.io/master/images/itowns_logo_300x134.png)


I think we should not copy a part of the parent README here but just explains that is contains all other itowns functionalities and link to the parent README of the repo and to the doc instead.

gchoqueux added enhancement 🚀 help wanted 📢 chore 🚚 refactor 🏭 architecture 🏬 labels Mar 21, 2024

gchoqueux force-pushed the subpackages branch 8 times, most recently from 9431214 to e23cfef Compare March 27, 2024 17:31

gchoqueux requested a review from Desplandis March 27, 2024 17:54

gchoqueux force-pushed the subpackages branch 5 times, most recently from e5ec2ea to b2308ff Compare April 3, 2024 12:02

gchoqueux marked this pull request as ready for review April 3, 2024 13:54

gchoqueux force-pushed the subpackages branch 6 times, most recently from ea0116d to 962c99d Compare April 9, 2024 14:56

Desplandis requested review from jailln and ftoromanoff April 15, 2024 14:16

Desplandis requested changes Nov 7, 2024

View reviewed changes

gchoqueux force-pushed the subpackages branch 3 times, most recently from d578888 to 1bd17ac Compare November 7, 2024 17:51

gchoqueux mentioned this pull request Nov 12, 2024

Obsolete prepare script #2462

Open

gchoqueux force-pushed the subpackages branch 12 times, most recently from f0ddd49 to 61e950f Compare November 13, 2024 17:22

gchoqueux force-pushed the subpackages branch from 61e950f to af45df1 Compare November 15, 2024 13:43

jailln reviewed Nov 26, 2024

View reviewed changes

packages/Geographic/README.md Show resolved Hide resolved

packages/Geographic/README.md Show resolved Hide resolved

package.json Outdated Show resolved Hide resolved

gchoqueux force-pushed the subpackages branch 2 times, most recently from ac0a1ad to 451da03 Compare November 28, 2024 12:16

gchoqueux force-pushed the subpackages branch 2 times, most recently from 6f3ada2 to d4616ba Compare November 28, 2024 14:34

gchoqueux added 2 commits November 28, 2024 15:34

chore(architecture): monorepo structure

5066e79

fix(test): increase time out

b1d0ba1

gchoqueux force-pushed the subpackages branch from d4616ba to b1d0ba1 Compare November 28, 2024 14:34

jailln reviewed Dec 3, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Monorepo structure #2300

Monorepo structure #2300

gchoqueux commented Mar 21, 2024 •

edited

Loading

gchoqueux commented Mar 27, 2024 •

edited

Loading

Desplandis commented Mar 27, 2024 •

edited

Loading

Desplandis left a comment •

edited

Loading

Desplandis Nov 7, 2024

gchoqueux Nov 13, 2024

gchoqueux commented Nov 13, 2024

jailln left a comment

gchoqueux commented Nov 28, 2024

Desplandis commented Dec 2, 2024 •

edited

Loading

jailln left a comment

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024

jailln Dec 3, 2024


		The geographic package provides utilities for handling coordinates, ellipsoids, extents and rotations across different coordinate systems.

		* [Coordinates](http://www.itowns-project.org/itowns/docs/#api/Geographic/Coordinates) : A Coordinates object (geodetic datum), defined by a [crs] and three values.


		// change projection system to EPSG:2154

		// defs EPSG:2154 crs (visiting epsg.io to get new definition and export to Proj4js)

		@@ -0,0 +1,131 @@
		![iTowns](https://raw.githubusercontent.com/iTowns/itowns.github.io/master/images/itowns_logo_300x134.png)

Monorepo structure #2300

Are you sure you want to change the base?

Monorepo structure #2300

Conversation

gchoqueux commented Mar 21, 2024 • edited Loading

Description

Motivation and Context

Code movement

dependencies

node scripts

gchoqueux commented Mar 27, 2024 • edited Loading

Desplandis commented Mar 27, 2024 • edited Loading

Desplandis left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gchoqueux commented Nov 13, 2024

jailln left a comment

Choose a reason for hiding this comment

gchoqueux commented Nov 28, 2024

Desplandis commented Dec 2, 2024 • edited Loading

jailln left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gchoqueux commented Mar 21, 2024 •

edited

Loading

gchoqueux commented Mar 27, 2024 •

edited

Loading

Desplandis commented Mar 27, 2024 •

edited

Loading

Desplandis left a comment •

edited

Loading

Desplandis commented Dec 2, 2024 •

edited

Loading